Automated Annotation Workflow¶
This workflow uses the auto_annot tools from besca to newly annotate a scRNAseq dataset based on one or more preannotated datasets. Ideally, these datasets come from a similar tissue and condition.
We use supervised machine learning methods to annotate each individual cell utilizing methods like support vector machines (SVM) or logistic regression.
First, the traning dataset(s) and the testing dataset are loaded from h5ad files or made available as adata objects. Next, the training and testing datasets are corrected using scanorama, and the training datasets are then merged into one anndata object. Then, the classifier is trained utilizing the merged training data. Finally, the classifier is applied to the testing dataset to predict the cell types. If the testing dataset is already annotated (to test the algorithm), a report including confusion matrices can be generated.
import besca as bc
import scanpy as sc
import pkg_resources
test load datasets with scvelo¶
Apparently the scv loader makes sure the adata objects are all in comparable format whereas the sc loader loads them as is.
adata_test = bc.datasets.Kotliarov2020_processed()
adata_test_orig = bc.datasets.Kotliarov2020_processed()
adata_train1 = bc.datasets.Granja2019_processed()
Concatenation does not lead to errors when the scv loader is used.
adata_train_list = [adata_train1]
Parameter specification¶
Give your analysis a name.
analysis_name = 'auto_annot_pubimage_trainGtestK' # The analysis name will be used to name the output files
Specify column name of celltype annotation you want to train on.
celltype ='dblabel' # This needs to be a column in the .obs of the training datasets (and test dataset if you want to generate a report)
Choose a method:
- linear: Support Vector Machine with Linear Kernel
- sgd: Support Vector Machine with Linear Kernel using Stochastic Gradient Descent
- rbf: Support Vector Machine with radial basis function kernel. Very time intensive, use only on small datasets.
- logistic_regression: Standard logistic classifier iwth multinomial loss.
- logistic_regression_ovr: Logistic Regression with one versus rest classification.
- logistic_regression_elastic: Logistic Regression with elastic loss, cross validates among multiple l1 ratios.
method = 'logistic_regression'
Specify merge method. Needs to be either scanorama or naive.
merge = 'scanorama' # We recommend to use scanorama here
Decide if you want to use the raw format or highly variable genes. Raw increases computational time and does not necessarily improve predictions.
use_raw = False # We recommend to use False here
You can choose to only consider a subset of genes from a signature set or use all genes.
genes_to_use = 'all' # We suggest to use all here, but the runtime is strongly improved if you select an appropriate gene set
Column names need to be standardised so the function knows which columns to compare.
adata_train_list[0].obs["dblabel"] = adata_train_list[0].obs.celltype3
adata_test.obs["dblabel"] = adata_test.obs.celltype3
adata_test_orig.obs["dblabel"] = adata_test_orig.obs.celltype3
adata_test.obs.dblabel.unique()
adata_train_list[0].obs.dblabel.unique()
adata_test.var.dtypes
adata_train_list[0].var.dtypes
Correct datasets (e.g. using scanorama) and merge training datasets¶
This function merges training datasets, removes unwanted genes, and if scanorama is used corrects for datasets.
adata_train, adata_test_corrected = bc.tl.auto_annot.merge_data(adata_train_list, adata_test, genes_to_use = genes_to_use, merge = merge)
Train the classifier¶
The returned scaler is fitted on the training dataset (to zero mean and scaled to unit variance). The scaling will then be applied to the counts in the testing dataset and then the classifier is applied to the scaled testing dataset (see next step, adata_predict()). This function will run multiple jobs in parallel if if logistic regression was specified as method.
classifier, scaler = bc.tl.auto_annot.fit(adata_train, method, celltype, njobs=10)
Prediction¶
If in addition to the most likely class you would like to have all class probabilities returned use the following function. (This is only a sensible choice if using logistic regression.)
adata_predicted = bc.tl.auto_annot.adata_pred_prob(classifier = classifier, scaler = scaler, adata_pred = adata_test_corrected, adata_orig = adata_test_orig, threshold = 0.0)
Output¶
The adata object that includes the predicted cell type annotation can be written out as h5ad file.
adata_predicted.write('./adata_predicted_trainGtestK.h5ad')
If the testing dataset included already a cell type annotation, a report can be generated and written, which includes metrics, confusion matrices and comparative umap plots.
adata_predicted.obs
adata_predicted = bc.st.clustering(adata_predicted, '.')
%matplotlib inline
sc.settings.set_figure_params(dpi=90)
bc.tl.report(adata_pred=adata_predicted, celltype=celltype, method=method, analysis_name=analysis_name,
train_datasets=adata_train_list, test_dataset=adata_test_orig, merge=merge, use_raw=False,
genes_to_use=genes_to_use, remove_nonshared=True, clustering='leiden', asymmetric_matrix=True)
